Method and apparatus for retrieving label

ABSTRACT

Disclosed is a method for retrieving a label in a portable terminal. The method includes obtaining a label image photographed through a camera, extracting characters included in the label image and recognizing the extracted characters, detecting at least one label including the recognized character from a label database including multiple labels and information on the multiple labels and constituting a preliminary label candidate group including said at least one label, detecting an image characteristic of the label image, detecting at least one label having an image characteristic, which is similar with the detected image characteristic, from the preliminary label candidate group, and constituting a final label candidate group, and providing each of said at least one label included in the final label candidate group and detailed information corresponding to each of said at least one label.

PRIORITY

This application claims priority under 35 U.S.C. §119(a) to a KoreanPatent Application entitled “Method And Apparatus for Retrieving Label”filed in the Korean Industrial Property Office on Sep. 11, 2009 andassigned Serial No. 10-2009-0085937, the contents of which areincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to a portable terminal, and moreparticularly to a method and an apparatus for retrieving a label byusing a portable terminal.

2. Description of the Related Art

As a result of the development of low-power, high efficiency, cheaphardware, the domain of the portable terminal has been recently expandedfrom simple voice calls to an application field employing image storage,reproduction, data communication, image process technologies, etc. Basedon the development of the hardware technologies, the portable terminalcan include a high-definition camera and provide various functions usingthe camera. For example, the portable terminal provides a function ofcharacter recognition, barcode recognition, face recognition, etc. usingan image photographed through the camera.

In the meantime, according to the increase of the interest of people inwine, the demands for wine have increased. The annual domesticconsumption of wine increases every year, and a 25% average annualgrowth rate has been recorded. Further, production of domestic wine andthe consumption of imported wine have also increased, and the domesticwine market is expected to increase to $413,000,000 in 2010 according toresearch. As such, according to the increase of the interest in and theconsumption of wine, a wine portal site providing wine information hasbeen created, and online sales of wine, various offline activities, andcommunity activities have been actively run.

As the interest in wine sharply increases, various services related towine are being generated, and the most general service is to provideinformation on wine through a web-service. However, user demand varies,and users desire to receive the information on wine anytime and anywherein a necessary time.

The tastes, kinds, and prices of wine vary according to a producer, aproducing region, a species of grapes, a producing year, etc., and suchwine information serves as a reference for the selection of wine. Alabel of wine is uniquely designed according to the wine information, sothat a consumer can discriminate a wine through the wine label andobtain information on the wine. However, even if a producer or a brandof wine is the same, but the kinds of the wine are different, the labelsmay have the similar designs, so that it requires the professionalknowledge so as to discriminate between wines.

Therefore, if it is possible to recognize a wine label, retrieve acorresponding wine label by using a recognized result, and provide auser with wine information using a portable terminal, the user, who doesnot have knowledge of wine, can receive wine information anytime andanywhere.

According to a conventional method for providing wine information, abarcode is attached to a wine label, the barcode is recognized, and thena wine is discriminated, so as to provide corresponding information ofthe wine. However, in the conventional method, the barcode has to bedirectly attached to a bottle of wine, so that additional expense isincurred and an additional barcode reader for reading the barcode isrequired, thereby failing to allow the general user to easily access themethod.

SUMMARY OF THE INVENTION

Accordingly, the present invention has been made to solve theabove-stated problems occurring in the prior art, and the presentinvention provides a method and an apparatus capable of retrieving awine label and providing information on wine anytime and anywhereregardless of a place or a time.

Further, the present invention provides a method and an apparatuscapable of retrieving a corresponding wine label by using a wine labelimage obtained through a camera included in a portable terminal.

Furthermore, the present invention provides a method and an apparatuscapable of rapidly and accurately providing information on wine.

In accordance with an aspect of the present invention, there is provideda method for retrieving a label of a label retrieval apparatus, themethod including obtaining a label image photographed through a camera;extracting characters included in the label image and recognizing theextracted characters; detecting at least one label including therecognized character from a label database including multiple labels andinformation on the multiple labels and constituting a preliminary labelcandidate group including said at least one label; detecting an imagecharacteristic of the label image; detecting at least one label havingan image characteristic, which is similar with the detected imagecharacteristic, from the preliminary label candidate group, andconstituting a final label candidate group; and providing each of saidat least one label included in the final label candidate group anddetailed information corresponding to each of said at least one label.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of the presentinvention will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating the construction of a labelretrieval apparatus according to an embodiment of the present invention;

FIG. 2 is a block diagram illustrating the construction of a characterrecognition unit according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating the operation of a label retrievalapparatus according to an embodiment of the present invention;

FIGS. 4A and 4B are flowcharts illustrating the operation of a characterrecognition unit according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating the operation of an image comparisonunit according to an embodiment of the present invention;

FIG. 6 illustrates a label image according to an embodiment of thepresent invention;

FIGS. 7A to 7E illustrate a process of a character area detectionaccording to an embodiment of the present invention;

FIG. 8 illustrates a binarization result according to an embodiment ofthe present invention;

FIG. 9 illustrates an inclination according to an embodiment of thepresent invention;

FIGS. 10A and 10B illustrate an example of an inclination correctionaccording to an embodiment of the present invention;

FIGS. 11A and 11B illustrate an example of a segmentation of a characterarea segmentation according to an embodiment of the present invention;

FIGS. 12A and 12B illustrate a result of a character area segmentationaccording to an embodiment of the present invention;

FIG. 13 illustrates a process of a combined character generationaccording to an embodiment of the present invention;

FIG. 14 illustrates a recognition result according to an embodiment ofthe present invention;

FIG. 15 illustrates a characteristic of a character design according toan embodiment of the present invention; and

FIG. 16 illustrates a screen showing a label retrieval result accordingto an embodiment of the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE PRESENT INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings. In the following description,the same elements will be designated by the same reference numeralsalthough they are shown in different drawings. Further, in the followingdescription of the present invention, a detailed description of knownfunctions and configurations incorporated herein will be omitted when itmay make the subject matter of the present invention rather unclear.

There are various types of wine and different wine labels. However, if aproducer or a brand of wine is the same, the labels have similardesigns, so that a beginner cannot easily discriminate between differentwines. Further, there are multiple wine labels, which may include thesame terms, but have entirely different designs, to be discriminated asdifferent labels, so that it is more difficult to discriminate a labeland accurately recognize wine information included in a correspondinglabel.

The present invention allows a user to rapidly and accurately recognizea label by using a photographed image of a label (e.g. a wine label),which includes characters and is in a specific design, retrieving acorresponding label, and then providing the user with relevantinformation.

To this end, the present invention extracts characters from a labelimage obtained through photographing and recognizes the character,retrieves the recognized characters in a label database, detectsreference label images including the same character, and constructs apreliminary label candidate group including the reference label images.Then, the present invention detects an image characteristic representinga characteristic of a label design from the label image and detectsreference label images including the similar image characteristic fromthe preliminary label candidate group, to provide the user with thedetected reference label images. With regard to this, it may be possibleto provide various information related to the detected reference labelimages, for example, a product, company information, etc. correspondingto the corresponding label.

Hereinafter, in order to help the understanding of the presentinvention, a description will be given on an example of discriminating awine label among various types of labels. However, the present inventioncan be similarly applied to a discrimination of other labels, e.g.labels of various drinking water, clothing trademarks, companytrademarks, etc.

According to an embodiment of the present invention, a portable terminalobtains wine label images through a digital camera mounted in theportable terminal, recognizes characters indicated in a wine labelthrough a pre-processing, retrieves the label image in a preliminarylabel candidate group obtained through the retrieval of the recognizedcharacter, and then provides the user with information on wine in asequence of the most similar label image.

When the portable terminal recognizes a wine label attached to a bottleof wine, due to a curved surface of the bottle, a wine label image hasan irregular characteristic that an illumination shape is changed as thewine label image retreats farther in a left or a right direction from acenter of the wine label. Therefore, it is not easy to accuratelysegment a background and a character area in a character area detectionprocess and a binarization process. As a result, the character areabecomes distorted, so that it may be difficult to recognize a shape ofthe character. Such a result renders the unique information of thecharacter unclear and results in an important reason for mis-recognitionin the recognition process. Therefore, it is necessary to analyze astructural characteristic of a curved label attached on a bottle ofwine, and detect and binarize a character area having an exactstructure.

A method for recognizing a character is generally classified into threemethods. The first method considers an entire word as a single unitwithout a character segmentation process and recognizes a character. Thesecond method generates only a single segmentation result having a highreliability in a character segmentation process. The third methodtransfers multiple segmentation results to a recognition process anddetermines a word segmentation result while recognizing the characters.

The first method, i.e. the word unit recognition method, can obtain ahigh recognition performance only with a characteristic of a widepattern distance between words and usually results in non-accurateperformance, so that it may be usefully used in an application field inwhich the number of used words are extremely limited. However, there area lot of used words in most of the application fields. Therefore, apattern distance between the words is close and a lot of similar wordsare included, so that the use of the precise characteristic close to thecharacter recognition is required, thereby failing to use the advantageof this method.

The second method refers to the external segmentation method, and canachieve a fast processing speed, so that it is widely used in therecognition of a printed word, in which characters are relatively andeasily segmented into a character. However, when the characters areincorrectly segmented, it is impossible to correct the charactersegmentation. Therefore, an error is generated during the charactersegmentation process and may fatally influence the entire recognitionprocess.

The third method refers to the internal segmentation method, andreceives the results of the segmentation of multiple characters, therebyimproving a character recognition performance. However, multiplerepetitions of character recognitions are usually required, so thatthere is a big burden on the processing speed. In the meantime, most ofthe character recognition devices have various limitations for highrecognition performance, so that it is inconvenient to use the characterrecognition device.

Therefore, in order to improve the accuracy of the characterrecognition, the highly reliable character segmentation method, whichmaximally decreases the burden on the processing speed in therecognition process, while using the internal segmentation method, isrequired.

An image used in character recognition is formed with character pieceareas. Therefore, a single character can be segmented into severalcharacter pieces, and the segmentation of the single character into theseveral character pieces can be performed based on a binarization imageand a location value of each of the character pieces.

Therefore, the present invention generates combined characters based ona binarization image and a location value of each of the characterpieces, measures a similarity between the generated combined charactersand reference characters, and extracts an optimum character combination,and performs a post-processing, to improve the recognition performanceof the characters.

A result obtained through recognizing and retrieving characters includedin a wine label is generated as a candidate group including a singlewine label or multiple wine labels. In order to retrieve the wine labelthat is most similar with a wine label image included in the candidategroup, the present invention uses image characteristics obtainable froma wine label.

FIG. 1 illustrates a label retrieval apparatus according to the presentinvention. The label retrieval apparatus can be included in a portableterminal, e.g. a mobile phone, a PDA, and the like. Referring to FIG. 1,the label retrieval apparatus includes a retrieval unit 10, a characterrecognition unit 20, an image comparison unit 30, a camera 40, labeldatabase 50, a memory unit 60, a display unit 70, and a key input unit80.

The key input unit 80 includes a shutter key for controlling aphotographing of the camera 40, various functional keys, number keys,etc. Further, the key input unit 80 outputs key data corresponding tovarious key inputs, which are generated when a user presses any key, tothe retrieval unit 10.

The memory unit 60 stores programs for processing and controlling of theretrieval unit 10, reference data, various renewable storage data, etc.,which are provided to a working memory of the retrieval unit 10.Further, the memory unit 60 stores a preliminary label candidate groupand a final label candidate group.

The label database 50 includes reference wine label images correspondingto various wine labels, label character strings included in each of winelabels, and detailed information on wine corresponding to each of thewine labels. The detailed information includes a kind of correspondingwine, a producing area, a producing year, a producing region, history,etc. Further, the label database 50 stores image characteristics of eachof the reference wine label images. The label database 50 can also bepart of the memory unit 60.

The camera 40 outputs, according to a control of the retrieval unit 10,an image generated through photographing a subject to the characterrecognition unit 20 and the image comparison unit 30. That is, when theuser photographs a wine label with the camera 40, a wine label image isobtained and the obtained wine label image is output to the characterrecognition unit 20 and the image comparison unit 30.

The character recognition unit 20 wavelet transforms, according to acontrol of the retrieval unit 10, the wine label image so as to detect acharacter area from the input wine label image and analyzes frequencytransform components of a horizontal line, a vertical line, and adiagonal line. A part having a large frequency change among the threefrequency transform components corresponds to a character area. Thecharacter recognition unit 20 detects a character area and then performsa binarization, which segments the detected character area into acharacter area and a background area. Therefore, the character arearepresents a character itself in the label image, and an adaptivethreshold algorithm can be applied to the binarization. The characterrecognition unit 20 segments a binarized character area based on each ofthe characters and recognizes each of the characters preferably by usinga neural network, and constructs at least one candidate character stringby using the recognized characters. Further, the character recognitionunit 20 compares the candidate character strings with wine labelcharacter strings registered in the label database 50 and outputs a winelabel character string having the highest similarity as a finalrecognition result, to the retrieval unit 10.

The retrieval unit 10 controls the general operation of the labelretrieval apparatus. The retrieval unit 10 retrieves at least one winelabel including the recognized wine label strings in the label database50 by using the final recognition result input from the characterrecognition unit 20, and constructs a preliminary label candidate groupincluding reference wine label images corresponding to the retrievedwine label. Then, the retrieval unit 10 controls the image comparisonunit 30 in order to retrieve a reference wine label image similar withthe wine label image obtained through the photographing in thepreliminary label candidate group.

The image comparison unit 30 extracts image characteristics of the winelabel image input by the camera 40. At this time, the extracted imagecharacteristics include a size characteristic of the character areaextracted by the character recognition unit 20, a color characteristicof a wine label image, and a design characteristic of a character.Further, the image comparison unit 30 calculates a similarity betweenthe extracted image characteristics with image characteristics of thereference wine label images included in the preliminary label candidategroup, constructs a final label candidate group including reference winelabel images having the high similarity, and outputs the final labelcandidate group to the retrieval unit 10.

The retrieval unit 10 detects the reference wine label images includedin the final label candidate group and information of the correspondingwine from the label database 50 and displays the detected reference winelabel images and the information of the corresponding wine on thedisplay unit 70, thereby providing the user with a wine label retrievalresult and the wine information.

FIGS. 3 to 5 illustrate an operation of the label retrieval apparatusincluding the above elements. FIG. 3 is a flowchart illustrating thegeneral label retrieval process of the label retrieval apparatusaccording to an embodiment of the present invention, FIGS. 4A and 4B areflowcharts illustrating the operation of the character recognition unit20 according to an embodiment of the present invention, and FIG. 5 is aflowchart illustrating the operation of the image comparison unit 30according to an embodiment of the present invention.

When the user requests a wine label retrieval through the key input unit80, the wine label retrieval apparatus activates the camera 40. The userphotographs a wine label attached to a bottle of wine to be retrieved byusing the camera 40 included in the label retrieval apparatus.

The label retrieval apparatus obtains a wine label image as shown inFIG. 6 in step 101 of FIG. 3. The camera 40 outputs the label image tothe character recognition unit 20 and the image comparison unit 30.Then, the retrieval unit 10 controls the character recognition unit 20,extracts characters from the label image, and recognizes an extractedcharacters in step 103. The corresponding operation of the characterrecognition unit 20 will be described with reference to FIGS. 2, 4A, and4B.

FIG. 2 is a diagram illustrating the construction of the characterrecognition unit 20 according to an embodiment of the present invention.Referring to FIG. 2, the character recognition unit 20 includes acharacter area detection module 21, a character segmentation module 22,and a character recognition module 23.

Referring to FIGS. 4A and 4B, when the label image is input through thecamera 40, the character recognition unit 20 obtains the label image instep 201. Then, in order to successfully detect characters included inthe wine label image, the character recognition unit 20 wavelettransforms the label image, analyzes a directional component of a highfrequency area, expands a candidate character area, and detects anaccurate character area by using a grouping and a vertical histogram anda horizontal histogram. Then, the character recognition unit 20binarizes the detected character area by using an adaptive boundaryalgorithm.

Therefore, the character area detection module 21 of the characterrecognition unit 20 wavelet transforms a gray scale component of thelabel image for the detection of the character area in step 203. In step205, the character area detection module 21 performs the wavelettransform, and then generates a candidate character area by using adirectional component of a character area including the High Frequency(HF) component, i.e. a Vertical (V), a Horizontal (H), and a Diagonal(D) directional component, in the photographed label. The grouping ofthe areas can extend the character area according to the directionalcomponents by applying three different sub masks based on its vertical,horizontal, diagonal characteristics.

With regard to this, it is assumed that a representative character ofthe wine in the wine label is approximately located in a center of thewine label image, and an example of used algorithm is represented inTable 1.

TABLE 1 1 step: computing mean value in HFV, HFH, and HFD detectingdirectional component(V,H,D) if(mean × C<coef) img[ ][ ] = 255; elseimg[ ][ ] = 0; 2 step: region grouping HFV: 1×11 HFH: 11×1 HFD: 3×5 3step: text region detection If(V∩(HUD)) textregion; else background;

FIGS. 7A to 7E illustrate a detection result of the character areaaccording to an embodiment of the present invention. FIG. 7A is anoriginal of the wine label image, FIG. 7B shows a horizontal directionalexpansion of the candidate character area extracted from the wine labelarea of FIG. 7A, FIG. 7C shows a vertical directional expansion of thecandidate character area extracted from the wine label area of FIG. 7A,FIG. 7D shows a diagonal directional expansion of the candidatecharacter area extracted from the wine label area of FIG. 7A, and FIG.7E shows a result of the detection of the finally derived candidatecharacter area by using FIGS. 7B and 7C by the algorithm of Table 1.

The character area detection module 21 analyzes a characteristic of avertical area histogram and a horizontal area histogram of the candidatecharacter area detected through the analysis of the high frequencycomponents obtained through the wavelet transforming and detects anactual character area. Then, the character area detection module 21segments the actual character area detected through an application ofthe adaptive boundary algorithm into a background area and a characterarea. The segmented character area corresponds to an area constitutingan actual character. For example, the segmented character arearepresents an area constituting “BLUE NUN”.

The general characteristic of illumination of a curved surface image,such as a bottle of wine, is bright in a central area and becomes darkeras it retreats to a left or a right area. Therefore, in the presentinvention, the adaptive threshold algorithm is applied in order toperform the binarization. An example of a result of the binarizationprocessing is shown in FIG. 8.

Returning to FIG. 4A, the character segmentation module 22 of thecharacter recognition unit 20 corrects an inclination of the characterarea detected by the character area detection module 21 in step 207, andsegments the character in step 209.

The correction of the inclination in the segmentation-based characterrecognition method is very important. The mis-applied inclinationcorrection distorts an input character string enough to not be used.Such a distortion derives an incorrect character segmentation andfinally results in recognition failure.

The inclination correction related to the character recognition can bebasically realized through extracting straight line components from theentire input character strings and then obtaining the inclinations ofthe extracted components. The present invention uses a horizontal runfor the extraction of the straight line components.

FIG. 9 illustrates an inclination of a defined section according to anembodiment of the present invention. Referring to FIG. 9, if a length ofa run in horizontal runs forming a binary character area is longer thanan average run length, a corresponding run is deleted 353. Then, theremaining runs are connected, to form a single section 354. The sectionincluding one run, i.e. section 354 including a single run 351 isdetermined to have no special information, so that the section includingthe single run 351 is ignored, and an inclination 352 of section 354including at least two runs is measured and corrected. At this time,section 354 including many runs is determined to have more information,so that weight is given to the inclination as much as the number ofruns.

If the number of total runs is n, the number of total sections is m, thenumber of runs in each section is r_(k), and an inclination of eachsection is Θ_(k), an average inclination of a straight line componentΘ_(avg) is expressed by Equation (1).

$\begin{matrix}{{\Theta_{avg} = {\sum\limits_{k = 1}^{m}\;\frac{r_{k} \times \Theta_{k}}{n}}},{k = 1},\ldots\mspace{14mu},m} & (1)\end{matrix}$

The character segmentation module 22 corrects the binarized characterarea by using an average inclination obtained by Equation (1). Examplesof the binarized character area corrected as much as the averageinclination are shown in FIGS. 10A and 10B. FIG. 10A is an original ofthe binarized character area and FIG. 10B is the inclination-correctedcharacter area.

When the inclination correction has been completed, the charactersegmentation module 22 derives segmentation candidate points by using acharacteristic of a contact point between syllables for the segmentationof basic characters in step 209 of FIG. 4A. If a contact portion betweenEnglish letters is formed in a valley shape, there is a high probabilityin that such a contact portion corresponds to the segmentation candidatepoint. Using such a feature, when at least two higher runs are includedin a horizontal run structure, the midpoint between two higher runs isset as a segmentation point.

The segmentation points presented in the extracted character area inFIG. 10B are shown in FIG. 11A. However, referring to FIG. 11A, it canbe seen that unnecessary segmentation points have been found. Theunnecessary segmentation points correspond to pixels, of which upper orlower sections are closed with respect to the segmentation points, orlong pixels extending in the Y-axis direction with respect to thesegmentation points. Therefore, the character segmentation module 22removes the unnecessary segmentation points and segments the characterarea based on the remaining segmentation points. The state in which theunnecessary segmentation points have been removed, is shown in FIG. 11B.

In the present invention, the character string included in the extractedcharacter area is excessively segmented within an optimum level, andthen the segmented character areas are re-combined in the recognitionprocess as necessary. However, if the character area is segmented intotoo many character areas, a large burden is created on the processingspeed during the recognition process. Therefore, when the segmented areaincluding a single character among the segmented areas is segmented,through recombining of the segmented areas, it is possible to reduce theprocessing time during the recognition process.

Therefore, the character segmentation module 22 classifies the segmentedarea into a noise area, a character piece area, and an effectivecharacter area based on an average number of pixels of each of thesegmented areas. The effective character area includes any character andcan increase a high character recognition rate. The noise area is anarea other than the areas including the character or an area having asize smaller than a reference size. The noise area does not increase ahigh correctness in the character recognition and may deteriorate therecognition rate, so that the noise area is removed. The character piecearea corresponds to an effective area constituting any character, but isan area separate from the corresponding character. That is, thecharacter piece area refers to an area where the remaining area of acharacter related to the character piece area is mostly located on avertical line. Therefore, when the number of connection pixels of eachsegmented area is equal to or less than a threshold range, the characterpiece area is re-combined with another effective character area existingon the same vertical line. If another effective character area is notlocated on the same vertical line, the character piece area isdetermined to be a noise area, and is removed.

Steps 211 to 221 of FIGS. 4A and 4B illustrate a characterre-combination process. In step 211 of FIG. 4A, the charactersegmentation module 22 identifies if the number of pixels of thesegmented area is greater than or equal to a maximum threshold. Themaximum threshold is determined based on an average number of pixels ofeach of the segmented areas, and for example, may be 30% of the averagenumber of pixels. When the number of pixels of the segmented area isgreater than or equal to the maximum threshold, the charactersegmentation module 22 determines the corresponding area to be aneffective character area in step 213 and proceeds to step 221 in FIG.4B.

When the number of pixels of the segmented area is less than a maximumthreshold in step 211, the character segmentation module 22 identifiesif the number of pixels of the segmented area is less than a minimumthreshold in step 215. When the number of pixels of the segmented areais less than the minimum threshold as a result of the identification,the character segmentation module 22 determines the corresponding areaas a noise area and removes the corresponding area in step 219, andproceeds to step 221 in FIG. 4B. The minimum threshold may be 10% of theaverage number of pixels. If the number of pixels of the segmented areais greater than or equal to the minimum threshold and less than themaximum threshold, the character segmentation module 22 determines thecorresponding area as a character piece area in step 217 and proceeds tostep 221 in FIG. 4B. The character segmentation module 22 recombines thecharacter piece area and the effective character area and finallycompletes the character segmentation in step 221.

Then, the character recognition module 23 of the character recognitionunit 20 generates every available combined character, i.e. temporalcombined characters, by using the finally segmented character areas,i.e. the segmented character areas, in step 223.

According to the present invention, when the total number of segmentedcharacter areas is M, M is larger than N, which is the number ofcharacters constituting the character string included in the label.Hereinafter, each of M segmented character areas is indicated as Si, andi is a natural number from 0 to M.

A word recognition algorithm according to the present invention is basedon two assumptions given on the segmented character areas. First, Sirepresents a part of a single character or an entire character. Second,a single character maximally includes α number of image pieces. Thefirst assumption implies that Si cannot include partial images of two ormore of the characters. This assumption is based on the word recognitionalgorithm, which segments a character area constituting a characterstring included in a label into several character segments and thenretrieves a combination constituted with optimum groups corresponding tocharacters within the character string.

FIGS. 12A and 12B show examples, in which a character area correspondingto the character string, “Volnay”, included in a wine label image isexcessively segmented into 8 image pieces. The word recognitionalgorithm according to the present invention has a goal of finding anoptimum combination, which includes “V” formed with first three imagepieces among 8 image pieces and “olnay” formed with each of theremaining characters.

The character recognition module 23 retrieves every availablecombination of the segmented character areas segmented by the charactersegmentation module 22. The recognition method through the excessivesegmentation combines the segmented character area in a sequence of thesegmentation as shown in FIG. 13 and extracts a recognition result. Inorder to compare the recognition result with the character to berecognized, the character recognition module 23 has to select acombinable character set and generate the combined characters.

Here, the combinable character set is as follows: When the recognitionresult is r, and the recognition result of the combined character isr_((a,b)) (where, “a” denotes a piece with which the combination beginsand “b” denotes the number of pieces used in the combination), thecombination of r_((0,3)) and r_((1,1)) is not allowed. The reason isthat the combination has been performed using the 0, 1^(st), and 2^(nd)pieces in r_((0,3)), so that r_((1,1)), which has used the 1^(st) piece,cannot be used for the combination. Therefore, such a charactercombination satisfying the above condition is generated and used in apost-processing.

In order to generate the combination with M segmented character areas,first, information on the maximum number and the minimum number of wordsto be recognized should be given.

In the present embodiment, only the words corresponding to a name of thewine label are limited to a recognition subject, so that the minimumnumber of words is 1 and the maximum number of the words is 3.Therefore, the character recognition module 23 derives availablecombinations by using the information on the number P of words, andgenerates a combination matrix as information on the combinations. Asize of the combination matrix can be obtained by an inductive method asfollows.

The number of available combinations with the last piece, SM, is 1.Since the available combinations in SM-1 are {SM, SM-1}, {GM, M-1}, thenumber of available combinations is 2. With regard to this, GM and M-1are the characters in which SM is combined with SM-1. The availablecombinations in SM-2 are {SM-2, SM, SM-1}, {SM-2, GM, M-1}, {GM-1, M-2,SM}, and {GM, M-1, M-2}, so that the number of available combinations is4. The available combinations in SM-3 are {SM-3, SM-2, SM, SM-1}, {SM-3,SM-2, GM, M-1}, {SM-3, GM-1, M-2, SM}, {SM-3, GM, M-1, M-2}, {GM-2, M-3,SM, SM-1}, {GM-2, M-3, GM, M-1}, and {GM-1, M-2, M-3, SM}, so that thenumber of available combinations is 7. Here, the number of availablecombinations in SM-3 is the sum of the number of available combinationsin SM-2, the number of available combinations in SM-1, and the number ofavailable combinations in SM. Therefore, the number of availablecombinations in S1 is the sum of the number of available combinations inS2, the number of available combinations in S3, and the number ofavailable combinations in S4. The temporal combined characters aregenerated in a number equal to the number of combinations correspondingto each Si.

Further, the combination matrix can be obtained through the dynamicprogramming method in the same way as the method for calculating thenumber of combinations. The combination matrix in SM-3 can be obtainedthrough adding SM-3, GM-2, M-3, GM-1, M-2, and M-3 to a front of SM-2,SM-1, and SM combination matrixes, respectively, so that it is possibleto minimize the time required for the additional matrix calculation.Further, the present word recognition algorithm recognizes the wordthrough matching each of the combinations formed by the dynamicprogramming method with each word in a dictionary, so that the number ofcombinations generated within the given number of image pieces isgreatly related to a word recognition rate of the present method.

The character recognition module 23 derives an optimum combination fromthe combinations found during the above processes. Each row of thecombination matrix represents a combination that can be matched with theword having a size of P. The character recognition module 23 previouslyperforms the character recognition of each element Gn of thecombination, i.e. the generated combined character, before matching thesingle combination Si with the word having a size of N in the dictionaryDn. With regard to this, elements to be recognized in the group Gn arelimited to characters on a unigram dn(N) including the characterslocated in the n^(th) location of the words included in the dictionaryDn.

As the number of segmented character areas increases, the processingtime increases by geometric progression. Therefore, the large number ofsegmented character areas results in difficult recognition of cursivecharacters extracted from the wine label by the portable terminal. Inorder to solve the above problem, the character recognition module 23compares the temporal combined characters according to the recognitionresult with a recognition result of the temporal combined characters andextracts an optimum combined character candidate group in step 225 inFIG. 4B.

That is, in step 225, the character recognition module 23 extracts adirectional segment feature of each of the temporal combined charactersgenerated according to the present invention, performs in advance an MLP(Multi Layer Perceptron) process based on the extracted directionalsegment feature, temporarily recognizes a corresponding temporalcombined character as a specific character, evaluates a correctness ofthe specific character with an SVD (Singular Value Decomposition)process, and forms the combined character candidate group. With regardto this, as the similarity between the combined character and thespecific character is higher, the correctness is higher, and thecombined characters belonging to the combined character candidate groupare the combined characters having a predetermined degree of thecorrectness ranking or higher.

The MLP process used in the recognition process derives only therecognition result, but does not calculate the similarity between theinput image, i.e. the combined character, and the recognition result.However, there is an advantage in that the recognition process is verysimple and it is possible to minimize the processing time. In thederivation of the recognition result, the SVD process can extract thesimilarity, but requires a large quantity of calculations. In order tosolve such a problem, the present invention appropriately combines theMLP and the SVD processes.

That is, when it is assumed that a characteristic of an input image usedin MLP is F₍₁₎ and a characteristic stored in the database correspondingto the recognition result C is F_((C)), the character recognition module23 evaluates F₍₁₎ and F_((C)) with SVD and measures the similarity.Then, the character recognition module 23 calculates an average of thesimilarities of the characters used in the combination and determinesthe average similarities as a similarity of the combined character.

With regard to this, it has been noted that the time required for thecharacter recognition when MLP is combined with SVD may be shorter thanthat required for the character recognition when only SVD is used.

It is assumed that the English letter is recognized, a processing timeof MLP is T_((M)), and the processing time of the SVD is T_((S)). Inthis case, when MLP is combined with SVD according to the presentinvention, the time T required for the character recognition can beexpressed by Equation (2).T=T _((M))+(T _((S))/26)  (2)

A corresponding alphabet letter is determined among 26 alphabet lettersaccording to the result of MLP with respect to any combined character,and above any combined character and the determined alphabet letter areevaluated with SVD, to obtain the similarity, so that time T isexpressed by Equation (2).

The processing time is T_((S)) when only SVD is used, so that ifT_((M))<(T_((S))/26)*25 is maintained, it is possible to curtail thetime required for the final character recognition.

Since the directional segment feature is extracted by considering theleft-right directional component and the diagonal directional componentof each pixel in the character area, it is possible to extract a featurerobust to a skew of the character and the processing process is simple,so that the processing time can be minimized. Further, in order tomeasure the diagonal directional feature in the extraction of thedirectional segment feature, the image is rotated 45°. The rotation ofthe image may cause an increase in processing time and a change in thefeature of the image, so that the value of the diagonal directionalcomponent is approximated using a trigonometrical function withoutrotation of the image. The processing of the diagonal segment featurecan include following steps.

First, a horizontal directional component (Hxy) with respect to eachpixel (x,y) is obtained. Also, a vertical directional component (Vxy)with respect to each pixel (x,y) is obtained. Then, a contribution (Dxy)is obtained by the horizontal directional component (Hxy) and thevertical directional component (Vxy).

In order to extract the feature without changing the shape and the sizeof the character area in the character extraction process, a nonlinearsegmentation having a feature independent of the size of the characterarea is performed. The nonlinear segmentation involves complicatedprocessing in comparison with a linear segmentation, but it is possibleto omit the process of the conversion of the character area and overcomedistortion caused by the change of the size of the character area.

The nonlinear segmentation process can be divided into the followingsteps. First, a horizontal directional histogram and a verticaldirectional histogram of the character area are obtained. Then, the sumof each of the histograms is obtained and the obtained sum is segmentedby a size of N of a mesh to be segmented, to obtain a threshold of eachof the histograms. Next, each of the histograms is segmented accordingto the threshold and an original image is segmented by a horizontalsegmentation value and a vertical segmentation value.

In the nonlinear segmentation, the segmentation sizes are variouslyextracted, to be compared and analyzed, and the highest performance hasbeen represented when a mesh of 5*7 is used and the number ofcharacteristic vector dimensions is 5*7 (35).

Further, in order to evaluate the similarity for the evaluation of thecorrectness of the combined character with the recognition result, SVD(Singular Value Decomposition) and cosine similarity are used. The SVDprocess in linear algebra is one of the important methods for thedecomposing of a rectangular matrix, and is widely used in signalprocessing and a statistics field. Further, SVD may be a generation of aspectrum theory of a matrix with respect to any rectangular matrix.Using the spectrum theory, it is possible to decompose an orthogonalrectangular matrix to a diagonal matrix based on a unique value.

It is assumed that a matrix M is an m×n matrix including elements of areal or complex number set K. At this time, M can be expressed by amultiplication of three matrixes as given in Equation (3).

Here, U is an m×m unitary matrix, Σ is an m×n matrix, the diagonalelements of which have a non-negative number and the remaining elementsof which have a value of 0, and V* is a conjugate matrix and representsan n×n unitary matrix. As such, the multiplication of the three matrixesis referred to as the SVD of M. In general, the larger value between Σ iand i is first written and in this case, Σ is solely determinedaccording to M.M=UΣV*  (3)

The cosine similarity between the singular value obtained through theSVD of the characters included in the dictionary and the characteristicvector of the input character is calculated. As the cosine similarityhas a smaller value, the character is determined to have the highersimilarity. The cosine similarity can be expressed by Equation (4).

$\begin{matrix}{{\cos\;\theta} = \frac{X \cdot Y}{{X}{Y}}} & (4)\end{matrix}$

In FIG. 4B, when the combined character candidate group has beencompletely extracted through the above process, the characterrecognition module 23 combines the characters in the combined charactercandidate group and generates multiple candidate character strings instep 227, and proceeds to step 229. At this time, the combined charactercandidate group can include a predetermined number of combinedcharacters in a sequence of the higher similarity among the multiplecombined characters. In step 229, the character recognition module 23compares the candidate character strings with the label database andoutputs the candidate character string having the highest similarity asa final recognition result.

FIG. 14 illustrates an example of the combined character candidate group401, the candidate character strings 403 generated by the combinedcharacter of the combined character candidate group 401, apost-processing result 405 representing a comparison process between thecandidate character strings 403 with the label database 50, and apost-processing result summing and arrangement list 407, which sums thepost-processing results and then represents the results in a sequence ofthe higher ranking.

When the character recognition has been completed through the processesof FIGS. 4A and 4B, the character recognition unit 20 transfers theresult of the character recognition to the retrieval unit 10. Returningto FIG. 3, the retrieval unit 10 searches the label database by usingthe candidate character string input as a result of the finalrecognition result from the character recognition unit 20 in step 105.Then, the retrieval unit 10 constitutes a preliminary label candidategroup including the wine labels including the candidate character stringin step 107. This is because that according to the characteristic of thewine label, even if the wine labels include the same character string,but have different designs, the wine labels can be classified asdifferent wine labels.

Then, the retrieval unit 10 controls the image comparison unit 30 andrecognizes an image characteristic of the label image in step 109. Theretrieval unit 10 retrieves a label having the high similarity in thepreliminary label candidate group by using the image characteristicdetected by the image comparison unit 30 while linking with the imagecomparison unit 30 and constitutes a final label candidate group in step111.

The operation of the image comparison unit 30 will be described withreference to FIG. 5. As illustrated in FIG. 5, the image comparison unit30 detects a size of the character area in the label image in step 301.The shapes and the sizes of the characters included in the wine labelare diverse. Therefore, the entire character area itself detected duringthe character recognition process can be the characteristic for thediscrimination of the image. The size of the label image can bedifferentiated according to a photographing distance, so that a ratio ofa horizontal length of the character area to a vertical length of thecharacter area is defined as a size characteristic. With regard to this,the horizontal length of the character area is a horizontal length ofthe entire character string included in the wine label and the verticallength of the character area is also the vertical length of the entirecharacter string included in the wine label. For example, the ratio ofthe horizontal length to the vertical length of the character area 501of FIG. 15 is the size characteristic.

Then, the image comparison unit 30 detects a color characteristic of thelabel image in step 303. The characteristic, which is most widely usedin the contents based image retrieval, is the color. The color in an RGBchannel has a great change in the color value according to light, sothat there are many cases in which the similarity of the color iscalculated as low. Therefore, it is preferred to use the color in a HSVchannel, which is less insensitive to light than the RGB channel.Equations (5), (6), and (7) are used when the RGB colors are convertedto the HSV (hue, saturation and value) colors, respectively.

$\begin{matrix}{H = {\cos^{- 1}\left\{ \frac{\frac{1}{2}\left\lbrack {\left( {R - G} \right) + \left( {R - B} \right)} \right\rbrack}{\sqrt{\left( {R - G} \right)^{2} + {\left( {R - B} \right)\left( {G - B} \right)}}} \right\}}} & (5) \\{S = {1 - {\frac{3}{R + G + B}\left\lbrack {\min\;\left( {R,G,B} \right)} \right\rbrack}}} & (6) \\{V = {\frac{1}{3}\left( {R + G + B} \right)}} & (7)\end{matrix}$

The present invention uses a color histogram in the derivation of thecolor characteristic. A value of the H component in the histogram of theHSV color space is segmented into 18 sections, the S component issegmented into 3 sections, and the V component is segmented into 8sections, so that the color histogram of a total of 432 dimensions isdefined.

The image comparison unit 30 counts the number of pixels correspondingto each of the histograms, among the pixels included in the label image,and extracts the color characteristic.

Then, the image comparison unit 30 detects a design of the character instep 305. In order to extract a shape characteristic of the character,the image comparison unit 30 has to first detect edges of thecharacters. The sizes of the characters are diverse, so that in order tonormalize the sizes, the size of the character area is converted into apredetermined size, e.g. 256×256 size (in pixels), as shown in FIG. 15,and then extracts the edges. In order to extract the characteristic ofthe character design, a horizontal projection histogram in the edgedetection image is used. When the horizontal component is used, aquantity of the histogram is changed according to the number and thekind of the characters, so that it is possible to measure thesimilarity.

When the image characteristic extraction from the label image has beencompleted, the image comparison unit 30 compares the extracted imagecharacteristic with an image characteristic of each of the referencewine label images included in the preliminary label candidate group andretrieves the reference label images having the high similarity, toconstitute the final label candidate group.

The present invention uses the sum of the similarities of threecharacteristics as an image characteristic similarity. The firstcharacteristic (the size of the character area) uses the ratio of thesizes of the character areas as the similarity, and the secondcharacteristic (color histogram) and the third characteristic (edgehistogram) uses the Euclidean distance as the similarity.

The calculation of the image characteristic similarity is expressed byEquations (8) to (11).

In the following equations, Q represents the image characteristicextracted from the label image and D is an image characteristic value ofthe reference label image included in the label database. Also, rrepresents a size characteristic, and CH and EH refer to the colorhistogram and the edge histogram, respectively. In order to normalizethe similarity value of each of the color characteristics and the designcharacteristic, the color histogram is obtained by calculating histogramvalues segmented by the total number of pixels of the image. Further,the edge histogram is obtained by calculating the Euclidian distance ofhistogram values segmented by 255, which is the number of columns in asingle row, and then segmenting again the calculated Euclidian distanceby 256, which is the number of rows. W represents the weight of eachcharacteristic. sim_(r) represents a similarity of a sizecharacteristic, sim_(c) represents a similarity of a colorcharacteristic, sim_(c) represents a similarity of a designcharacteristic, and sim represents a similarity of a final imagecharacteristic. As the value calculated by Equations (8) to (11) becomeslower, the similarity is higher.

$\begin{matrix}{{sim}_{r} = {Q_{r} - D_{r}}} & (8) \\{{sim}_{c} = \sqrt{\sum\limits_{i = 1}^{432}\;\left( {\frac{Q_{{CH}_{i}}}{n \times m} - \frac{D_{{CH}_{i}}}{n \times m}} \right)}} & (9) \\{{sim}_{e} = \sqrt{\frac{\sum\limits_{i = 1}^{256}\;\left( {\frac{Q_{{EH}_{i}}}{256} - \frac{D_{{EH}_{i}}}{256}} \right)^{2}}{256}}} & (10) \\{{sim} = {{w_{1}{sim}_{r}} + {w_{2}{sim}_{c}} + {w_{3}{sim}_{e}}}} & (11)\end{matrix}$

Returning to FIG. 3, when the image comparison unit 30 provides theretrieval unit 10 with the final label candidate group, the retrievalunit 10 provides the reference wine label images included in the finallabel candidate group and wine information corresponding to the winelabel through the display unit 70. This example is illustrated in FIG.16.

FIG. 16 illustrates a wine label retrieval result 603 with respect tothe photographed label image 601. In FIG. 16, the wine label retrievalresult 603 includes only reference wine label images detected in thelabel database 50. However, when the wine label image is selected, thecorresponding wine information is detected in the label database 50 andis displayed. Then, the multiple reference wine label images may bedisplayed in a sequence representing the higher similarity.

Accordingly, the present invention can retrieve wine labels by using aportable terminal, e.g. a mobile phone, a PDA (Personal DigitalAssistant), or the like, so that it is possible to retrieve the winelabel and provide wine information anytime and anywhere regardless of aplace and a time. Further, the present invention can retrieve thecorresponding wine label by using the wine label image obtained througha camera mounted in the portable terminal. Furthermore, the presentinvention can rapidly and accurately provide the wine information.

While the present invention has been shown and described with referenceto certain exemplary embodiments and drawings thereof, it will beunderstood by those skilled in the art that various changes in form anddetails may be made therein without departing from the spirit and scopeof the invention. For example, according to the present invention, thelabel database is included in the label retrieval apparatus, but can beincluded in a separated apparatus. In this case, the label retrievalapparatus accesses the label database, so as to obtain the necessaryinformation. Therefore, it would be appreciated by those skilled in theart that the scope of the present invention should be defined in theclaims and their equivalents, not by the embodiments of the presentinvention.

What is claimed is:
 1. A method for retrieving a label using a labelretrieval apparatus, the method comprising the steps of: obtaining alabel image photographed through a camera; extracting charactersincluded in the label image and recognizing the extracted characters;detecting at least one label including the recognized character from alabel database including multiple labels and information on the multiplelabels; generating a preliminary label candidate group including the atleast one label; detecting an image characteristic of the label image;detecting at least one label having an image characteristic, which issimilar to the detected image characteristic, from the preliminary labelcandidate group; generating a final label candidate group including theat least one label having the image characteristic, which is similar tothe detected image characteristic; and providing each of the at leastone label included in the final label candidate group and detailedinformation corresponding to each of the at least one label included inthe final label candidate group.
 2. The method as claimed in claim 1,wherein recognizing the extracted character comprises: wavelettransforming the label image and extracting a character candidate areaby using a horizontal frequency transform component, a verticalfrequency transform component, and a diagonal frequency transformcomponent; binarizing the character candidate area; segmenting thebinarized character candidate area into a character area including anactual character and a background area; detecting the character area;correcting an inclination of the character area; segmenting thecharacter area into multiple segmented character areas; generatingmultiple combined characters based on the multiple segmented characterareas; generating multiple candidate character strings based on by usingthe multiple combined characters; and comparing the multiple candidatecharacter strings with label character strings included in the labeldatabase and outputting a candidate character string having a highsimilarity as a final recognition result.
 3. The method as claimed inclaim 2, wherein in the step of correcting the inclination of thecharacter area, in multiple horizontal runs of the character area, whena length of a predetermined run is longer than an average length of arun, the predetermined run is omitted, remaining runs are connected andgenerated as a single segment, a segment including one run is determinedto have no special information to be ignored, and an inclination of asegment including at least two runs is measured and corrected.
 4. Themethod as claimed in claim 3, wherein segmenting the character area intothe multiple segmented character areas comprises: using a characteristicof the horizontal run to segment the character area into the multiplecharacter areas; when a number of pixels of the segmented area isgreater than or equal to a maximum threshold, determining the segmentedarea to be an effective character area; when the number of pixels of thesegmented area is less than the maximum threshold and greater than orequal to a minimum threshold, determining the segmented area to be acharacter piece area; when the number of pixels of the segmented area isless than the minimum threshold, determining the segmented area to be anoise area; and removing the noise area and combining the characterpiece area with the effective character area located on a same verticalline.
 5. The method as claimed in claim 4, wherein generating themultiple combined characters comprises: combining the segmentedcharacters and generating temporal combined characters; performing anMLP (Multi Layer Perceptron) process for each of the temporal combinedcharacters; recognizing the temporal combined character as a specificcharacter; evaluating a correctness of the specific character by usingan SVD (Singular Value Decomposition) process; and generating themultiple combined characters including combined characters having apredetermined similarity ranking or higher.
 6. The method as claimed inclaim 1, wherein detecting the image characteristic of the label imagecomprises: detecting a ratio of a horizontal length of the characterarea to a vertical length of the character area of the label image as asize characteristic; detecting a color characteristic of the labelimage; detecting edges of the characters included in the label image;and detecting a design characteristic.
 7. The method as claimed in claim1, wherein the label includes a wine label.
 8. A label retrievalapparatus comprising: a camera; a label database including multiplelabels and information on the multiple labels; a character recognitionunit for extracting characters included in a label image obtainedthrough the camera, recognizing the extracted characters, and outputtinga recognition result; a retrieval unit for detecting at least one labelincluding the recognized characters from the label database by using therecognition result input from the character recognition unit andgenerating a preliminary label candidate group including the at leastone label; and an image comparison unit for detecting an imagecharacteristic of the label image, detecting at least one label havingan image characteristic similar with the detected image characteristicfrom the preliminary label candidate group, and generating a final labelcandidate group including the at least one label having the imagecharacteristic similar with the detected image characteristic from thepreliminary label candidate group, wherein the retrieval unit provideseach of the at least one label included in the final label candidategroup and detailed information corresponding to each of the at least onelabel included in the final label candidate group.
 9. The labelretrieval apparatus as claimed in claim 8, wherein the characterrecognition unit comprises: a character area detection module forwavelet transforming the label image, extracting a character candidatearea based on a horizontal frequency transform component, a verticalfrequency transform component, and a diagonal frequency transformcomponent, binarizing the character candidate area and segmenting thebinarized character candidate area into a character area constituting anactual character and a background area, and detecting the characterarea; a character segmentation module for correcting an inclination ofthe character area and segmenting the character area into multiplesegmented character areas; and a character recognition module forgenerating multiple combined characters based on the multiple segmentedcharacter areas, generating multiple candidate character strings byusing the multiple combined characters, comparing the multiple candidatecharacter strings with label character strings included in the labeldatabase, and outputting a candidate character string having a highsimilarity as a final recognition result.
 10. The label retrievalapparatus as claimed in claim 9, wherein, in multiple horizontal runs ofthe character area, when a length of a predetermined run is longer thanan average run length, the character segmentation module deletes thepredetermined run, connects remaining runs with each other and generatesa single section, determines that a section including one run does nothave special information to ignore the section, and measures andcorrects an inclination of a section including at least two runs. 11.The label retrieval apparatus as claimed in claim 10, wherein thecharacter segmentation module segments the character area into multiplecharacter areas based on a characteristic of the horizontal run,determines the segmented area to be an effective character area when anumber of pixels of the segmented area is greater than or equal to amaximum threshold, determines the segmented area to be a character piecearea when the number of pixels of the segmented area is less than themaximum threshold and greater than or equal to a minimum threshold, anddetermines the segmented area to be a noise area when the number ofpixels of the segmented area is less than the minimum threshold, and thecharacter segmentation module deletes the noise area and combines thecharacter piece area with the effective character area located on a samevertical line, to segment the character area into the multiple segmentedcharacter areas.
 12. The label retrieval apparatus as claimed in claim11, wherein the character recognition module combines the segmentedcharacters and generates multiple temporal combined characters, performsan MLP (Multi Layer Perceptron) process for the multiple temporalcombined characters and recognizes a temporal combined character as aspecific character, evaluates a correctness of the specific characterbased on an SVD (Singular Value Decomposition) process, and generatesthe multiple combined characters including combined characters having asimilarity in a predetermined ranking or higher.
 13. The label retrievalapparatus as claimed in claim 12, wherein the image comparison unitdetects a ratio of a horizontal length of the character area to avertical length of the character area in the label image as a sizecharacteristic, detects a color characteristic of the label image, anddetects edges of the characters included in the label image and detectsa design character.
 14. The label retrieval apparatus as claimed inclaim 9, wherein the label includes a wine label.